Human collective intelligence as distributed Bayesian inference

Collective intelligence is believed to underly the remarkable success of human society. The formation of accurate shared beliefs is one of the key components of human collective intelligence. How are accurate shared beliefs formed in groups of fallible individuals? Answering this question requires a multiscale analysis. We must understand both the individual decision mechanisms people use, and the properties and dynamics of those mechanisms in the aggregate. As of yet, mathematical tools for such an approach have been lacking. To address this gap, we introduce a new analytical framework: We propose that groups arrive at accurate shared beliefs via distributed Bayesian inference. Distributed inference occurs through information processing at the individual level, and yields rational belief formation at the group level. We instantiate this framework in a new model of human social decision-making, which we validate using a dataset we collected of over 50,000 users of an online social trading platform where investors mimic each others’ trades using real money in foreign exchange and other asset markets. We find that in this setting people use a decision mechanism in which popularity is treated as a prior distribution for which decisions are best to make. This mechanism is boundedly rational at the individual level, but we prove that in the aggregate implements a type of approximate “Thompson sampling”—a well-known and highly effective single-agent Bayesian machine learning algorithm for sequential decision-making. The perspective of distributed Bayesian inference therefore reveals how collective rationality emerges from the boundedly rational decision mechanisms people use. Human groups have an incredible capacity for technological, scientific, and cultural creativity. Our historical accomplishments and the opportunity of modern networked society to stimulate ever larger-scale collaboration have spurred broad interest in understanding the problem-solving abilities of groups—their collective intelligence. The phenomenon of collective intelligence has now been studied extensively across animal species [1]; collective intelligence has been argued to exist as a phenomenon distinct from individual intelligence in small human groups [2]; and the remarkable abilities of large human collectives have been extensively documented [3]. However, while the work in this area has catalogued what groups can do, and in some cases the mechanisms behind how they do it, we still lack a coherent formal perspective on what human collective intelligence actually is. There is a growing view of group behavior as implementing distributed algorithms [4, 5, 6, 7], which goes a step beyond the predominant analytical framework of agentbased models in that it formalizes specific information processing tasks that groups are solving. Yet this perspective provides little insight into one of the key features of human group cognition—the formation of shared beliefs [8, 9, 10, 11, 12, 13]. 1 ar X iv :1 60 8. 01 98 7v 1 [ cs .C Y ] 5 A ug 2 01 6 Individual-Level “Social Sampling” Mechanism Step 1. Choose an option to consider according to popularity. (Sampling according to social prior) Available Options